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ABSTRACT 



The need for an auxiliary performance measure for the 
design of constrained optimal controls for linear regula- 
tors is shown. Several auxiliary performance measures are 
compared, and the maximum of the absolute degradation over 
the admissible initial states is selected as the auxiliary 
performance measure. Computational algorithms which make 
extensive use of existing library subprograms are developed 
for the design of constrained optimal controls in those 
cases where the control vector is specified as constant or 
piecewise-constant linear feedback of the output vector. 
Numerical examples including a third-order system and a 
time-varying system are given to illustrate the applica- 
tions of the proposed algorithms. 
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INTRODUCTION 



I . 

The linear regulator is one of the most extensively 
studied and well-known problems of optimal control theory 
[1,2,3,41. The importance of linear regulators comes from 
the fact that many practical control problems can be formu- 
lated in the form of a linear regulator. In addition, the 
feedback form of the optimal solution to the problem is a 
very desirable feature. The practical application of the 
optimal solution to the linear regulator problem suffers 
from two serious drawbacks. One is the time-varying nature 
of the feedback gains, and the other is the need for exact 
physical measurements of the value of the state vector at 
every instant of time. In general, not the state vector of 
the system, but the output vector of the system is available 
for measurements. It is possible to reconstruct the state 
vector [ 5 ], or to obtain an approximation to the state vec- 
tor that is best in some sense when noise is present [6] ; 

however, in many cases, these schemes to obtain the state 
vector are not justified due to economic considerations. 

Since the mathematical formulation and solution of the 
optimal linear regulator problem by Kalman [ 2 ], the design 
of subop timal controls, which are easy to implement, yet 
not too inferior to optimal ones has attracted the atten- 
tion of several investigators. 

Schoenberger [ 7 ] has given solutions to a class of 
problems in which the initial conditions are known. He has 
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also presented a procedure for minimizing the expected 
value of the performance measure when the probability dis- 
tribution of the initial conditions is known. 

Meditch [8] has proposed an approximate method of de- 
coupling complex systems which simplifies the computation 
of the time-varying feedback gains. 

Koivuniemi [ 9 ] has specified the form of the feedback 
control and has determined the unknown parameters to mini- 
mize the maximum degradation of the system performance over 
the admissible initial states. 

Rekasius [ 10 ] has proposed a control law that minimizes 
the maximum (with respect to all initial states) relative 
deviation in the value of the performance measure with re- 
spect to the optimal performance measure. Although his 
method avoids some of the difficulties related to minimax 
problems, it is impractical for systems of higher than the 
second order. 

In Kleinman and Athans' [ll] formulation of the sub- 
optimal linear regulator problem, easily realizable time 
functions are used to generate the subop timal time-varying 
feedback gains. The unknown parameters are determined to 
minimize the expected value of the performance measure over 
the admissible initial states, which are assumed to be uni- 
formly distributed on a unit hyper sphere. 

Kleinman, Fortmann and Athans [12] have given an ap- 
plication of Kleinman and Athans 1 [ll] approach. They 
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have presented an algorithm for the numerical solution of 
the problem, for, the case where the system is time-invari- 
ant and the suboptimal feedback gains are constrained to be 
piecewise constant. 

Levine and Athans C 1 3 3 have also used an average per- 
formance measure, but they have let the control vector to 
be a time-varying function of the system- output rather than 
the system state. 

Salmon [14] has proposed an algorithm which can be ap- 
plied to the design of controllers for systems with unknown 
parameters, including initial conditions. In this method, 
the controller parameters are found in such a way that the 
maximum of either the system performance measure or the 
so-called performance sensitivity with respect to the un- 
known system parameters is minimized. The performance sen- 
sitivity is defined as the percentage or the absolute in- 
crease in the performance measure from that of an ideal con- 
troller caused by the unknown system parameters. The ideal 
controller is defined as the controller which is capable of 
measuring unknown system parameters and generating its own 
parameters accordingly. 

In this thesis, after a brief review of the theory of 
optimal control of linear regulators, the need for an aux- 
iliary performance measure for the formulation of the con- 
strained optimal control problem is shown and several aux- 
iliary performance measures are compared using a simple 
second-order example. Then the constrained optimal control 
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problem is formulated and a numerical method of solution 
for the case of constant feedback gains is given. Next, an 
approximate, but very practical, solution to the problem 
for the case of piecewise-cons tant feedback gains is pro- 
posed. Numerical examples that illustrate the properties 
of the techniques are given. 
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II. OPTIMAL CONTROL OF LINEAR REGULATORS 



Consider the linear, time-varying system 



x(t) = A ( t) x ( t) + B(t)u(t) 



y.(t) = C (t)x(t) 



( 1 ) 

( 2 ) 



where x(t) is the n-dimensional state vector, u(t) is the 
m-dimensional unconstrained control vector, y.(t) is the 
r-dimensional output vector, and A(t), I3(t) and C^(t) are 
time-varying matrices of dimensions n x n, n x m, and r x n, 
respectively. Determine the control u * ( • ) that minimizes 
the performance measure 



j (Xq' t o ,u(*) ) = \ x T (t f )Hx(t f ) 






[x T (t)g(t)x(t) + u T (t) R(t)u(t) ]dt 



(3) 



where t and t_ > t are the initial and final times, re- 
o f o 

spectively, Xq = 2£.(t Q ) is the initial state, H is a real 
symmetric positive semi-definite matrix, £(t) is a time- 
varying real symmetric positive semi-definite matrix, R(t) 
is a time-varying real symmetric positive definite matrix, 
and the final state x(t f ) is free. 
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The solution to this problem is well-known [l, 2, 3, 4-3. 
The optimal control exists and is given by 



u*(t) = - R 1 (t)B T (t)K(t)x(t) 

= - F*(t)x(t) (4) 

where K(t) is the unique symmetric positive definite solu- 
tion of a matrix differential equation of the Riccati type 
given by 

K(t) = - K(t)A(t) - A T (t)K(t) - &(t) 

+ K(t) B(t) R _1 (t) B T (t)K(t) (5) 

with the boundary condition 



K ( t f ) = H . 



( 6 ) 



The state of the optimal system is the solution of 



x(t) = [A( t) - B(t)F*(t) 3x(t) 



(7) 



with the boundary condition x(t Q ) = The optimal per- 

formance measure is given by 






u ( . ) = u * ( • ) 



1 T 

= tt x K(t ) x . 

2 — o — o—o 



(8) 
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The matrix K(t) has another interesting property. For ar- 
bitrary t ^ tf and x(t) 

J*(x(t),t) = \ x T (t)K(t)x(t) (9) 

Some comments on the results just given are in order: 
first, the optimal control is a linear, time-varying feed- 
back of the system states. (See Fig. 1). Second, the 

matrix Riccati differential equation, Eq. (5) , is a set of 
2 

n coupled, first-order, nonlinear differential equations 
which can be solved numerically beginning at t = t^ and in- 
tegrating backward in time to t = t using the boundary con- 
dition (6). Actually, only n(n+l)/2 of these equations 
must be solved because K(t) is symmetric. 

In implementing the controller block in Fig. 1, the 
most appealing scheme is to calculate K(t Q ) by integrating 
(5) backward off-line and then to calculate F*(t) on-line 
by integrating (5) in the forward direction using previous- 
ly calculated K(t Q ). However, this is not possible in the 
time-varying case because of the instability of (5) in the 
forward time direction [2]. Even if the system is time- 
invariant, on-line integration is not always desirable due 
to increased complexity of» the system. Another possibility 
is to provide storage for the elements of the feedback gain 
matrix F*(t) and to store the precalculated values. This 
is possible because the elements of F*(t) do not depend on 
the system state but depend only on time; therefore, 
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I 1 




a priori knowledge of the initial state is not necessary. 
However, this scheme suffers from the same drawback as the 
previous scheme, namely, increased complexity. 

Suppose that F (t) is readily available. Even then 
the realization of the optimal control, u*(t), requires com- 
plete knowledge of the system state x(t) • Reconstructing or 
estimating the state vector [5,6] from the knowledge of the 
output, may add additional complexity to the control- 

ler . 

In many instances it is possible to decrease the com- 
plexity of a control system considerably by constraining 
the controller configuration and allowing a small increase 
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in the system performance measure with respect to optimal. 
If the adjustable parameters of such a controller are de- 
termined in some optimal fashion, then the control signal 
is called a constrained optimal control. In the following 
sections the design of constrained optimal controls for 
systems which can be modeled in the form of linear regula- 
tors will be discussed. 
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III. FORMULATION OF THE CONSTRA INED 



OPTIMAL CONTROL PROBLEM 



For the system and the performance measure given by 
(1) , (2) and (3) consider the control law 

u(t) = - P(t)y(t) (10) 

where j?(t) is an m x r matrix which will be called the out- 
put-feedback gain matrix. The physical meaning of this con- 
trol is as follows: the control signal is generated from 

the system output vector rather than from the system state 
vector as in the optimal control. Substituting (2) in (10) , 
one obtains 



u(t) = - P(t)C(t)x(t) 
- - F(t)x(t) 



( 11 ) 



The value of the performance measure for the control law 
(11) is defined as 



J (x 



fc o^ ( 



)) = J(x 



t Q ,u(.)) 



u(0 = -P(*)z(*) 



( 12 ) 



If F(t) defined by (11) is not equal to F*(t) defined by 
(4) for all t £ t £ tf, then the control (10) is not op- 
timal. In general, 

J(2£o» t Q ,P(.)) s j*(x o ,t Q ) (13) 
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It is shown in the Appendix that 



J (x 
— o 




( 14 ) 



where V(t)^ is the symmetric, positive definite solution of 



the matrix differential equation 



V( t) = - V(t) [A ( t) - B(t)P(t)C (t) ] 



- [A ( t) - B(t)P(t)C(t) ] T V(t) 



- fi(t) - C T (t)P T (t)R(t)P(t)C(t) 



(15) 



with the boundary condition 



V(t f ) = H 



(16) 



From the point of view of implementation, the control (10) 
has distinct advantages over the optimal control: first, 

knowledge of the system state vector x(t) is not needed. 
Second, constraints may be imposed on P.(t) to make the 
structure of the control simple, e.g., P.(t) may be required 
to be constant or piecewise constant. 

A. NEED FOR AN AUXILIARY PERFORMANCE MEASURE 

It is reasonable to suggest that the elements of .P(t) 
be chosen so that the performance measure (12) is minimized. 



Note that V(t) depends on the values that P.(t) attains in 
the interval between t and tf . However this will be shown 
explicitly only when t = t as in Eq. (14) . 



1 
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and at the same time the constraints imposed on the elements 
of P.(t) are satisfied. However, there is one major diffi- 
culty in proceeding in this fashion: it is the need for a 
priori knowledge of the initial state of the system. It 
was pointed out in the previous section that although 
J*( 2 £ Q /t o ) is a function of the initial state, the optimal 
feedback gain matrix F*(t) is independent of the initial 
state. However, this is not the case with the control (10), 
in general. Not only J(2^f t Q /P (•)), but also the best out- 
put-feedback gain matrix, P^(t), depends on the initial state. 
If the initial state is known a priori, then it is possible 
to find optimal values for the elements of P.(t), for 
t Q ^ t ^ tf, satisfying constraints of whatever complexity 
that may have been imposed on them, by minimizing the per- 
formance measure at least numerically, if not analytically. 

If the initial state is not known, then one approach is to 
define an auxiliary performance measure which is independent 
of the initial state. Some concepts that will be needed in 
defining an appropriate auxiliary performance measure will 
be given first. 

The Absolute Degradation , denoted by AD, will be 
defined as 

AD(x q , t Q ,P(*) ) = J(x Q , t Q ,P(.) ) - J*(x o ,t Q ) , (17) 

that is, the absolute degradation is the increase in the 
performance measure with respect to the optimal value caused 
by application of a control given by (10) . 
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The Relative Degradation , denoted by RD, is given by 



RD(2W-(*)) ~ 



J(x , t , P(-)) - J*(x , t ) 
v — o o — v — Q* o' 

T* 




( 18 ) 



Both the absolute and relative degradations are always non- 
negative quantities. 

Assuming that the possible initial states may lie 
within a hyperspherical region, with center at the origin 
and having a specified radius, there is no loss of general- 
ity in further assuming that the set of admissible initial 
states, denoted by X, consists of all points which lie on 
the unit hypersphere. This is because, for any admissible 
x and any p ^ 0 , 



i.e. , the performance measure due to an initial state which 
is not on the unit hypersphere is a constant times the per- 
formance measure due to the corresponding point on the 
unit hypersphere. 

There are two basic approaches in choosing an auxiliary 
performance measure: one is the statistical approach, the 

other is the worst initial state approach. In the first ap- 
proach, it is assumed that the probability distribution of 
the initial state is known, and a statistical function of 
the performance measure that is independent of the initial 





( 19 ) 
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state is taken as the auxiliary performance measure. Several 
authors [7,11,12,13] use the expected value of the perform- 
ance measure over the admissible initial states as the aux- 
iliary performance measure. Normally, a simple probability 
density function, such as a uniform distribution is assumed. 
This makes the expected value of the performance measure a 
very simple function of the V matrix found from (15) . 

Several variations on this approach depending upon the con- 
straints imposed on the structure of the control and the 
assumed admissible initial states are given in the references 
cited above. 

In the worst initial state approach [9,10,14], the max- 
imum (worst) over the admissible initial states of any one 
of three basic quantities - performance measure, absolute 
degradation, and relative degradation - can be taken as the 
auxiliary performance measure. Therefore, there are four 
functionals of £(•) which can be used as the auxiliary per- 
formance measure. These are: 



J-L (P(-)) 


= !(.))) 
X €X 

— o 


(20) 


J 2 (£(♦)) 


= Max J(x ,t ,£(•) ) 

x e X 
— o 


(21) 


J 3 (£(•)) 


= Max AD (x ,t ,P(*)) 
X €X -O O - 

— o 


(22) 


J 4 (£(•)) 


= Max RD(x q , t Q ,£( • ) ) 

x € X 

— o 


(23) 
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The output -feedback gain matrix, which minimizes the 
auxiliary performance measure is called the constrained op- 
timal output-feedback gain matrix and is denoted by P^(t). 

The constrained optimal control, denoted by u (t), is given 

s 

by (10) with P(t) replaced by P. s ( t) . 

The performance measure resulting from the application 
of a constrained optimal control is called the constrained 
optimal performance measure and is denoted by ). 



B. A COMPARISON OF SEVERAL AUXILIARY PERFORMANCE MEASURES 
1. Example 1 

The purpose of this example is to illustrate the 
differences that result from the "use of various auxiliary 
performance measures. The method of obtaining the solutions 
will not be discussed, but rather the implications of each 
different result will be indicated. 

The system to be controlled is described by 



I 

JJ 
1 — 1 




0 1 




X x (t) 




0 




= 








+ 




_x 2 (t)_ 




0 0 




_ X 2 (t) _ 




1 


Y 1 ( t) 




1 0 




x x (t) 






y 2 (t) 




0 1 




x 2 (t) 


• 





(24) 



(25) 



The performance measure for optimal control is 

2 



J (x * u ( * ) ) 



[x, 2 (t) + u 2 (t)]dt . 
o 



(26) 
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The constrained optimal control is to be in the form 



u s (t) 




y x (t) 

y 2 (t) 



= ‘ [ p n P 12] 



(t) 

x 2 (t) 



(27) 



where and p 12 are constants that minimize the auxiliary 

performance measure. 

This problem was solved four times, using the four dif- 
ferent auxiliary performance measures given by (20) - (23) . 
The values obtained for p^ and p^ 2 for each case are tabu- 
lated in Table 1. The conclusion that can be drawn looking 
at that table is very important: The values obtained for 

the parameters of the constrained optimal control depend 
very much on the auxiliary performance measure used. 
Therefore, the proper choice of an auxiliary performance 
measure is very significant. The values that the several 
performance quantities attain as functions of the initial 
state are also different in each case. Since the system is 
only second order and the initial state is assumed to lie 
on the unit circle, just one quantity, namely the ratio of 
the two components of the initial state vec tor , x 2 ( t Q )/k.|(t o ) , 
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Auxiliary Per- 
formance Measure 


P 11 


P 12 


Max RD(x ,t ,P) 

x eX 
— o 


0.3055 


0.5932 


E {j(x ,t ,P) } 

x € X 
— o 


0.8096 


1.1668 


Max J(x ,t ,P) 

x €X ^ ° 

— o 


0.7312 


1.5375 


Max AD(x , t ,P) 

x CX 
— o 


0.5643 


0.9463 



Table 1. Constrained optimal feedback 
gains for different auxiliary 
performance measures used in 
Ex. 1. 



is enough to specify a particular initial state'*'. The op- 
timal performance measure J*, constrained optimal perform- 
ance measure J , absolute degradation AD, and relative de- 
gradation RD are plotted as functions of x 2 ( / x i ( t Q ) on 
one graph for each of the auxiliary performance measures in 
Figs. 2, 3, 4 and 5. 



1 

It does not make any difference whether x 2 (t 0 ) and x^(t Q ) 
are both in the first quadrant or both in the third quad- 
rant (similarly for the second and fourth quadrants) due to 
symmetry. 
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Fig. 2. Various performance values as 
functions of initial state. 
(Maximum relative degradation 
minimized. ) 




Fig. 3. Various performance values as 
functions of initial state. 
(Expected value of performance 
measure minimized.) 
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Fig. 4. Various performance values as 
functions of initial state. 
(Maximum performance measure 
minimized. ) 




Fig. 5. Various performance values as 
functions of initial state. 
(Maximum absolute degradation 
minimized. ) 
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For this example, it is clear that using the maximum 
relative degradation as the auxiliary performance measure 
(Fig. 1) is not very desirable. Although the maximum RD 
is less than 0.2 it amounts to a large absolute degradation 
for initial states where the optimal performance measure is 
already high. 

When it is assumed that the initial states are uniform- 
ly distributed on the unit hypersphere and the expected 
value of the performance measure is used as the auxiliary 
performance measure (Fig. 3) the relative degradation goes 
as high as 0.43. But this occurs for initial states where 
the optimal performance measure is quite low, and the ab- 
solute degradation does not amount to a large value. In 

this case the J curve is closer to the J* curve on the 
s 

average than the other three cases. But, unless separately 
calculated as in this example, nothing can be said about 
maximum absolute degradation which may be more than allow- 
able. 

Fig. 4 shows the case when the maximum J is used as 
the auxiliary performance measure. In this case the con- 
strained optimal control is almost as good as the optimal 
control when the constrained optimal performance measure is 
the maximum. In other words, the control is the best 
(closest to the optimal) when the initial condition is the 
"worst". This is a very desirable feature. However, it 
results in a very pessimistic design, i.e., the absolute 
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degradation for the other initial states becomes quite 
high. As far as the absolute degradation is concerned, al- 
lowing a small increase for the worst initial state may re- 
sult in a significant improvement for the other initial 
states, as is the case in Fig. 5. 

In the last case where the maximum absolute degrada- 
tion is used as the auxiliary performance measure (Fig. 5) , 
the constrained optimal performance measure as a function 
of the initial state is made as close to the optimal per- 
formance measure as possible by minimizing the maximum dif- 
ference between the two, i.e., the maximum absolute degra- 
dation. When this approach is used one is assured that the 
system performance will not be degraded more than a certain 
amount with respect to the optimal system performance no 
matter what the initial state is. 

2 . Some Comments on Example 1 

Example 1 has demonstrated that the choice of an ap- 
propriate auxiliary performance measure is a significant 
step in the design of constrained optimal controls for 
linear regulators. It is easy to reject the maximum rela- 
tive degradation as the auxiliary performance measure in 
almost all applications. After all, one would not want to 
get farther away from the optimal performance as the con- 
trol task becomes more demanding. The remaining three de- 
finitions for the auxiliary performance measure are all 
reasonable and each one of them can be used depending on 
the application. During the remainder of this work, the 
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maximum absolute degradation will be used as the auxiliary 
performance measure. The results obtained can be easily 
applied to the case where the maximum performance measure 
is used as the auxiliary performance measure. The case 
where the expected value of the performance measure is used 
as the auxiliary performance measure is easier to handle 
mathematically (especially when the probability distribu- 
tion of the initial state is assumed to be uniform on the 
unit hypersphere) and is treated elsewhere [7,11,12] . 

C. STATEMENT OF THE CONSTRAINED OPTIMAL CONTROL PROBLEM 

It is assumed that the system is described by the state 
and output equations 

x(t) = A ( t) x ( t) + B(t)u(t) * (1) 

y;(t) = C(t)x(t) (2) 

given previously. The performance measure for optimal con- 
trol is 



J(x q , t Q ,u(*) ) = x T ( fc f )ii x(t f ) 



r> fc f 



+ *j 



[x T ( t)£( t)x( t) + u T ( t) R ( t) u ( t) ]dt . (3) 



Let the set of admissible initial states be X, and let X 
consist of the points which lie on the unit hypersphere. 
Let 0 denote a specified class of real time functions. 
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The constrained optimal control problem is to find a 
control 

u ( t) = - P (t)x(t) (28) 

such that the elements of £ (t) are members of 0 and the 

s 

auxiliary performance measure 

J (£(•)) = Max AD(x,t ,£(•)) (29) 

a x ex ° ° 

— o 

is minimized. 

The value of the auxiliary performance measure corre- 
sponding to JP g (t) is denoted by J* , i.e., 

(£<•)> 

£(•) =£_(•) 

s 

= Min J (£(•)) . (30) 

£(-)cO 3 

The magnitude of J* is an indication of how close the con- 

cl 

strained optimal control is to the optimal control. How- 
ever, a normalized number is usually more meaningful in 
making comparisons; therefore, the Normalized Degradation 
of the system, denoted by ND, will be defined as 



ND 



Max J 

x eX 
— o 



<V t o ) 



(31) 
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ND is zero if the performance of the constrained optimal 
control is the same as the performance of the optimal con- 
trol for all initial states; otherwise, it will be a posi- 
tive number. 
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IV. SOLUTION OF THE CONSTRAINED 



OPTIMAL CONTROL PROBLEM 



A. CONSTANT OUTPUT-FEEDBACK GAIN MATRIX 

In this section, the constrained optimal control prob- 
lem will be solved for the case where the elements of P^(t) 
are constrained to be constant in the interval between t 

o 

and t^ . 

1. Development of the Method 

It is desired that the constant matrix P be found 

— s 

such that 



J (P ) 
a v — s 



J 



* 

a 



= Min 
P 



Max 

X € X 

— o 



AD (x , t ,P) 
o o — 



(32) 



This is a minimax problem, and there are many difficulties 
associated with the solution of such problems: first, in 

general, an iterative numerical procedure in which a com- 
plete maximization process is required for each minimiza- 
tion step is necessary [9]; second, no matter how smooth 

the function AD(x , P) is in both x and P, the function 

—i o — — o — 

J (P) = Max AD(x ,t ,P) (33) 

a x fX -o o 

— o 

is not, in general, differentiable on the elements of _P. 

In particular, it may not be differentiable in P^ at the 
point yielding the minimum [14,15], Therefore, some 
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method which does not require the partial derivatives of 
J (P.) with respect to the elements of P is more suitable 

cl 

than, for example, a method in which the gradient vector is 
needed and the partial derivatives are evaluated by making 
small perturbations. 

It is possible to simplify the procedure consider- 
ably by noting that 



AD(Xo,t o ,P> 



= | So T Cv(t 0 .p) -Koglx, 



= ir X T W ( t ,p)x 
2 — o — o — — o 



(34) 



is a real quadratic form in x q » and that the following 
well-known [16,17] theorem about the extremal properties 
of the eigenvalues of a real quadratic form is applicable. 

Theorem: The global maximum of a real quadratic 

form on the unit hypersphere is equal to the largest eigen- 
value of the quadratic form, and moreover the corresponding 
eigenvector is the vector drawn from the origin to the 
point on the hyper sphere where the quadratic form achieves 
its maximum. Therefore 

Max AD(x ,t ,P) = 3 - X (P) (35) 

x ex 0 ^ 

— o 

where ( P.) is the largest eigenvalue of W(t 0 #P.) defined 
by (34), and the problem is to minimize X^ (_P) with respect 
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to the elements of P^ ; that is, to find the constant matrix 



P such that 
— s 




j Min X 1 ( P) 



(36) 



Such a formulation has several advantages over the formula- 
tion given by (32) . First, 



can be evaluated without the need to find the point where 
the maximum occurs. This is very important because (37) is 
evaluated many times during the minimization process, and 
any savings in the number of necessary computations is sig- 
nificant. Second, the programming required is extremely 
simple because very efficient methods are available for 
finding the largest eigenvalue of a real symmetric matrix, 
and many computer facilities have several standard subrou- 
tines that can be used for this purpose. However, it should 
be noted that a subroutine that finds all eigenvalues (and 
eigenvectors) of a general square matrix is not as suitable 
for this purpose as one which finds only the largest eigen- 
value of a real symmetric matrix. 



X-! (P) 



2 Max AD(x 
, — o 




(37) 



2 . Algorithm for Computing the Constant Output- 



Feedback Gain Matrix - Algorithm I 

a. Calculate K(t Q ) by integrating (5) backward 
with the boundary condition (6) . 

b. Guess starting values, for the entries of P. 

c. Find P^ that minimizes X^ (P) by using a sub- 
routine that preferably does not require the evaluation of 
the gradient. Such a subroutine will, in general, require 
the evaluation of X^ (£) many times. This can be accomplish- 
ed as follows:’ 

(1) Calculate V(t Q ,P) by integrating (15) back- 
ward with the boundary condition (16) , using the current 
value of P,. 

(2) Calculate W(t Q ,P) using (34). 

(3) Find the largest eigenvalue, X^(P), of 
W(t Q ,P} using a suitable subroutine. 

d. If the location of the point where the worst 
degradation occurs is desired, then find the eigenvector 
corresponding to X.. (£ ) . 

In applying this algorithm to the examples given 
in the rest of this thesis, the minimizations were accom- 
plished using a subroutine^ that performs the pattern 
search method of Hooke and Jeeves [18,19]; the largest 



1 

Subroutine DIRECT, NPS Computer Facility. 
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eigenvalues were found using a subroutine' 1 that utilizes 

Givens-Householder method; and integrations were performed 

2 

using a Runge-Kutta-Gill fourth-order method . 

3 . Application of Algorithm I - Example 2 

The purpose of this example is to illustrate the 
application of the described algorithm to a simple second- 
order system, and also to illustrate the importance of using 
the appropriate algorithm. 

The system to be controlled is described by 



x x (t) 




1 

O 

M 

1 




x 1 (t) 


_ x 2 (fc) _ 




-3 -1 




x 2 (t)_ 


""y 1 (t)~ 




1 0 




x 1 (t) 


_y 2 (t)_ 




0 1 




_ x 2 (t) _ 




u(t) 



(38) 



(39) 



The performance measure for optimal control is 



J(j£ o ,u(*) ) 



1 
2 J 



5 ff 
•T~ 



[x 1 2 (t) + x 2 2 (t) + u 2 (t)]dt. (40) 



The constrained optimal control is to be in the form 

Y x (t) 



u s (fc) = - [Pll P i2 ] 



y 2 (t) 



(41) 



1 

Subroutine GIVHO, see Ralston and Wilf [ 20 D . 

2 

Function RKLDEQ, NPS Computer Facility. 
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Expanding the Riccati 



where and p^ 2 are constants, 

equation (5) with 





0 1 




0 




1 0 


A = 




* B = 




, Q = 






-3 -1 




1 




0 1 



one obtains 

k i:L (t) = 6k 12 (t) - 1 + k 2 2 (t) 

*12 ( t) =k 12 ( t ) - ^n(t) + 3k 22 (t) +k 12 (t)k 22 (t) (42) 

k 22 (t) = 2[k 22 (t) - k 12 (t)] - 1 + k 22 (t) 

with the boundary conditions 

*11 (f9 = *12 (f) = *22 Cf) = 0 ' < 43 > 



Note that due to the symmetry of K(t) only three equations 
are needed. Expanding the equation (15) with 



£ = [ p u P12] and 




0 

1 



one obtains 



v lx (t) 



Vi2 ( t ) 



^ 22 (t) 



2 ( 3+p ll) V 12 ( t) “ 1 ~ Pxx 
(1+P i2 ) v i2 ( t) ~v 11 (t) + (3+p 11 

2 d +p 12) V 22 ^ ^ -2v 12 ^ ^ -1-p 



) v 22 
2 

12 



(t) Pxx p X2 

(44) 
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with boundary conditions 




Again due to symmetry only three equations are needed. 

The proposed algorithm gives the solution for the 
constant output- (or state) feedback gain matrix as 

P = [0.1537 0.5097] 

— s 

The normalized degradation (ND) is 0.00055 which shows that 
the constrained optimal control is almost as good as the 
optimal control. 

Koivuniemi [9] treats the same system as an ex- 
ample for the design of a constrained optimal controller 
which he calls a specific optimal controller. He uses the 
same auxiliary performance measure as here, but obtains a 
result which is quite different. He gives the constant out- 
put-(or state) feedback gain matrix as 

P = [0.69 0.76] 

— s 

Assuming that the initial states lie on the unit circle, 
the optimal performance measure J , constrained optimal 
performance measure J g , absolute degradation AD, and rela- 
tive degradation RD are plotted as functions of x 2 ( t 0 ) /x^(t Q ) 

using the P_ matrix obtained here and the £ matrix given 
s s 

in [ 9 ] in Figs. 6 and 7 respectively, for comparison. 

Clearly the result found here results in a much better 
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Fig. 6. Various performance values as 

functions of initial state - using 
feedback gains found in Example 2. 




Fig. 7. Various performance values as 

functions of initial state - using 
feedback gains given in Ref. 9. 
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performance for all initial states than the result given by 
Koivuniemi. This is also obvious from a comparison of the 
normalized degradation (ND) figures. It is 0.057 for 
Koivuniemi ' s design — much higher than 0.00055 obtained 
here . 

This example has illustrated the importance of 
using an efficient algorithm for the solution of the mini- 
max problem. Apparently, the algorithm given in [ 9 ] has 
failed at some point, resulting in a solution which is not 
as good as it could be for this example. 



and the constrained optimal control is specified in the 
form 



If the output of the same system (38) is given by 




y x (t) = [l 0 ] 



(46) 



x 2 (t) 



u s (t) = - p^y^t) , 



(47) 



then the expansion of (15) with 



= p^ and C = [l 0 ] 



gives 



Vn(t)= 2 (3+p 1]L ) v 12 (t) -1-p 



2 

11 



v 12 ^ - v-^2 ( t) v-^-^(t) ( 3+p ^ -^) ^22 ^ ^ 



( 48 ) 



v 22 (t)= 2v 22 (t) - 2v 12 (t) - 1 
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with boundary conditions (45) . Applying the same algorithm, 
one obtains 

£ s = P 11 = -°- 236 

which gives a normalized degradation (ND) of 0.137. 
Koivuniemi ' s result for this case is 

-s = P 11 = ~° * 25 

which agrees reasonably well with the result found here. 
Evidently, the algorithm given in [9] works in this case. 

B. PIECEWISE -CONSTANT OUTPUT-FEEDBACK GAIN MATRIX 

In this section, the constrained optimal control prob- 
lem will be solved for the case where the elements of £( t) 
are constrained to be piecewise-cons tant in the interval 

between t and t,. . 

o f 

1 . Development of the Method 

Suppose that there are N subintervals of time, each 
of duration At, during which the elements of P(t) are con- 
stant. Let 

t^ = t + iAt, i = 0,1, ... , N (49) 

and let 



P(t) = P ± , t ± * t < t i+1 , i = 0,1, 



, N-l 



(50) 
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Since is an m x r matrix and since there are N such con- 
stant matrices, one can minimize the auxiliary performance 
measure given by (29) with respect to the mrN elements which 
are to be determined and obtain the piecewise-cons tant P s (t). 
However if mrN is a large number, the minimization of the 
auxiliary performance measure with respect to such a large 
number of variables may be a very difficult and time-con- 
suming task. 

A much simpler, yet possibly more powerful and 
practical approach will be used here. This approach is in- 
spired by the Principle of Optimality of Bellman [ll. Sup- 
pose that the time t„ , has been reached and there remains 

N-l 

only one interval during which control can be applied. At 
this point, if x(t N _-^) were known a priori, then to deter- 
mine P^_^ would be a simple parameter optimization problem, 
and P, T . would be determined to minimize J(x(h T . ) , t„ 
However, x(t N-1 ) is not known a priori, therefore, one is 
again confronted with the problem of defining an auxiliary 
performance measure. Fortunately, the solution proposed 
for the constant-P case in the previous section can be ap- 
plied to this case also by simply taking as the ini- 

tial time and applying the Algorithm I. This gives the 
elements of Z N _-^ as well as K(t N _^), V(t N _^) and W(t N _^) . 
This solution for the last interval is equivalent to assum- 
ing that the admissible lies on the unit hyper- 

sphere (without loss of generality) and minimizing the max- 
imum absolute degradation with respect to the admissible 
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and 



x(t. T ,). Similarly, P 0 can be determined using t 

INI — ± 1N1 — Z N-2 

t^_^ as the initial and the final times respectively and 
using K(t N-1 ) and V(t N _ 1 ) obtained as a result of the pre- 
vious application as the boundary conditions in performing 
the integrations. The same procedure is applied to all 

subintervals until P , K(t ) and V(t ) are obtained at the 

— o — o — o 

last application. 

2 . Algorithm for Computing the Piecewise-Cons tant 
Output-Feedback Gain Matrix - Algorithm II 






a . 


Let i = N. 






b. 


Let tj. = t. and 


t = 


t. n . 




f l 


o 


1-1 


c . 


Apply Algorithm 


I to 


obtain 


d. 


If i = 1, go to 


f. 




e. 


Let i = i - 1, < 


JO to 


b. 


f. 


Stop. 







3 . Some Comments Concerning the Application of Algo- 
rithm II 

The optimal value for the auxiliary performance 
measure, J* , obtained by applying Algorithm II will be 

cl 

greater than the value that would be obtained if the mini- 
mization were accomplished at one step with respect to mrN 
elements as indicated earlier. However, it should be re- 
membered that in the latter case the "worst" initial state 
(the initial state that causes the maximum absolute degrada- 
tion) is assumed at time t Q only and not at the beginning of 
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each subinterval, whereas, in the method of solution pro- 
posed here, the constant output -feedback gain matrix for 
each subinterval is determined to minimize the maximum ab- 
solute degradation at the beginning of the subinterval. 

It is even possible that the normalized degradation 
may increase with a small increase in the number of subin- 
tervals. This is because the "worst" is assumed to occur 
more often and the parameter values that are chosen with 
this in mind may result in higher normalized degradation 
than the case where "worst" is assumed less often or just at 
the beginning. However, when this occurs, it still does not 
mean that increasing the number of subintervals is not ad- 
vantageous. This entirely depends on the application. If 
the system to be controlled is in an environment which is 
known very well and no disturbances are anticipated, then 
increasing the number of subintervals in the manner proposed 
here does not result in a "better" system, unless there is 
a decrease in normalized degradation. If piecewise-constant 
output-feedback gains are desired in such a case, then one 
should perform the minimization at one step with respect to 
all mrN elements as mentioned earlier. However, in practice 
many systems operate in environments where unpredictable 
disturbances can occur at any time. In those cases, increas- 
ing the number of subintervals even when the normalized de- 
gradation does not decrease is desirable, at least from a 
conservative design point of view. 
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When all of the states of a system are available 



as outputs and the number of subintervals approaches infin- 
ity, i.e., the constraints are relaxed, then one expects 
the constrained optimal control to approach the optimal con- 
trol and the normalized degradation to approach zero. This 
was the case in all the examples studied by the author. 

4 . Application of Algorithm II 
a. Example 3 

The purpose of this example is to illustrate 
the application of Algorithm II to a third-order system. 



The 


. 1 
system 


to be controlled 


is descr 
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x x (t) 




* 2 (t) 
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x 3 ^ 




Y x (t) 
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y 2 (t) 
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x 2 (t) 




_ y 3 ( t} .. 
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0 
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_x 3 (t)_ 



2 

2 u( t) 
-1 



(51) 



(52) 



The performance measure for optimal control is 



J (x Q , u ( . ) ) 



f 2 

o 



‘ Cx 1 (t)x 2 (t)x 3 (t) ] 







1 0 




x x (t) 


O 

rH 




CN 

X 


0 0 




- x 3 (t) _ 




1 

This system is treated in Ref. 12 where constant and 
piecewise-constant gains are computed using a different 
auxiliary performance measure. 
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The constrained optimal control is to be in the form 



u s U) = - Cp 11 (t) P 12 (t) P 13 ( t )^ 



y 1 (t) 

y 2 (t) 

y 3 (t) 



( 54 ) 



where [p-^^(t) p^ 2 (t) p^^(t)3 is a piecewise-constant out- 
put-feedback gain matrix. The matrices needed for the ap- 
plication of Algorithms I and II are: 





-1 
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0 


[V 
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0 
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A = 


0 


0 


2 


B = | 2 


C = 


0 


l 


0 




L° 


-2 


0 


-1 


1 


0 


0 


1 




2 


-2 


0~ 












2 = 


-2 


2 


0 


R = 2 


P = 


1 — I 


(t) 


P 12 




0 


0 


0 













( t) Pj^Ct)] 



and H = 0. This problem was solved for the one-subinterval 
(a constant feedback gain matrix) case using Algorithm I, 
and for two and ten-subinterval (piecewise-constant feed- 
back gain matrices) cases using Algorithm II. The elements 
of E| s (t) = P g (t)C^ and F* ( t) = R ^B T K(t) are plotted in Figs. 
8, 9, and 10 for each case. The normalized degradation is 
0.0356 for the one-subinterval case and decreases to the 
values 0.0135 and 0.0025 for the two and ten-subinterval 
cases, respectively. As the number of subintervals is 
further increased, the constrained optimal feedback gains 
become closer to the optimal feedback gains, and the 
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Fig. 8. Optimal and constrained optimal 
feedback gains for Example 3 — 
One subinterval. 
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Fig. 9. Optimal and constrained optimal 
feedback gains for Example 3 — 
Two subintervals. 
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Fig. 10. Optimal and constrained optimal 
feedback gains for Example 3 — 
Ten subintervals. 
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normalized degradation decreases. When the number of sub- 
intervals is fifty the normalized degradation is 0.0002. 
Clearly, for this example, one can get as close to the op- 
timal as desired by increasing the number of subintervals, 
b. Example 4 

The purpose of this example is to investigate 
and compare the effectiveness of using different states of 
a linear regulator to form the feedback control, and to 
provide another illustration of the applications of Algo- 
rithms I and II. The system to be controlled is again 
given by (51) and the performance measure for the optimal 
control by (53) . This time the output is a scalar given by 



y(t) 



c 



x x ( t) 
x 2 (t) 
x 3 (t) 



(55) 



where C is a 1 x 3 matrix, with only one element equal to 
one and the rest equal to zero. In other words, only one 
of the states is available as an output. The constrained 
optimal control is specified in the form 

u g (t) = - p(t)y(t) (56) 

where p(t) is the piecewise-constant output- feedback gain. 
Table 2 gives the normalized degradations for different 
outputs and for various numbers of subintervals. Clearly, 
C = [0 0 l] results in the best constrained optimal 
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control, i.e., x^Ct) is quite effective in forming the con- 
trol. Considering that the normalized degradation is 0.5653 
without any control (u(t)= 0) it can be seen that C = Cl 0 o] 
or C = [O 1 o] provide very little improvement over no con- 
trol. When C = Co 0 l], there is a small increase in the 
normalized degradation when the number of subintervals is 
increased from one to two. This was mentioned as a possi- 
bility previously. However, further increase in the number 
of subintervals decreases the normalized degradation again. 



G 


One 

Subinterval 


Two 

Subintervals 


Four 

Subintervals 


[i o o] 


0.5022 


0.5022 


0.5022 


Co l o] 


0.5252 


0.4767 


0.4419 


Co o i] 


0.0907 


0.0942 


0.0822 



Table 2. Normalized degradations for different outputs 
and number of subintervals for Example 4. 

c . Example 5 

In this example. Algorithms I and II are ap- 
plied to a second-order, time-varying system. The system 
is described by 
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The performance measure for optimal control is 

r 10 

J(Xo*u(*)) =\\ [200 x 1 2 (t) +2u 2 (t)]dt. 



(57) 1 



(58) 



(59) 



The constrained optimal control is to be in the form 



u s (t) 



[p il (t) p 12 (t)] 



y x (t) 

y 7 (t) 



(60) 



where Cp^^(t) p^ 2 (t)] is a piecewise-constant output-feed- 
back gain matrix. Expanding the equations (5) and (15) 
with 



1 

These equations represent the perturbed motion of the 
flight of an airplane in a vertical plane on a gliding path 
inside the equisignal zone of the glide radio beacon, with 
constant velocity. For more detail on the description of 
the system, see Example 1 of Ref. 21 where an optimal con- 
troller is designed for the system. 
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R = 2 and H = 0 and applying Algorithms I and II, the prob 

lem was solved for several cases with various numbers of 

subintervals. The elements of F (t) = £ (t)C^ and 

s s 

F*(t) = R ■*'B T K(t) are plotted in Figs. 11 and 12 for one 
and two-subinterval cases respectively. The normalized de 
gradation is 0.02 for the one-subinterval case and 0.0852 
for the two-subinterval case. This increase in the normal 
ized degradation with an increase in the number of subin- 
tervals was mentioned previously as a possibility in the 
comments following the description of Algorithm II. How- 
ever, as the number of subintervals is increased further, 
the normalized degradation decreases and approaches zero, 
e.g., it is 0.0205 for the ten-subinterval case and 0.0015 
for the fifty-subinterval case. This example shows that 
the application of Algorithms I and II to time-varying sys 
terns is as straightforward as the application to time- 
invariant systems. 
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Fig. 11. Optimal and constrained optimal feedback 
gains for Example 5 — One subinterval. 
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2 4 6 8 i-> 1 0 



Fig. 12. Optimal and constrained optimal feedback 
gains for Example 5 — Two subintervals. 
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V. CONCLUSIONS 



A . SUMMARY 

The problem of designing optimal controls for linear 
regulators under complexity constraints has been investi- 
gated. The need for an auxiliary performance measure was 
demonstrated and several candidates for the auxiliary per- 
formance measure were compared. The maximum of the abso- 
lute degradation over the admissible initial states was se- 
lected as the auxiliary performance measure. A method was 
developed and a computational algorithm (Algorithm I) was 
given to solve the resulting minimax problem for the case 
where the feedback gains are constrained to be constant. 

A conservative engineering approach inspired by the Princi- 
ple of Optimality was used for the solution of the same 
problem for the case where the feedback gains are constrain- 
ed to be piecewise-constant. It was assumed that the "worst" 
initial state could occur at the beginning of each subin- 
terval and the parameters were determined accordingly. A 
computational algorithm (Algorithm II) was given for this 
purpose . 

Both algorithms were programmed in FORTRAN IV for the 
IBM 360 in use at the NPS Computer Facility. Since the 
algorithms rely heavily upon existing, well-established 
library subprograms, the programming required was very 
simple, yet efficient and reliable. In none of the problems 
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solved did the results depend on the initial guess for the 
parameter values. For the minimization step of the algo- 
rithms a gradient scheme in which partial derivatives were 
evaluated by making small perturbations was also tried and 
gave the same results as the pattern search in all cases. 
The gradient algorithm, however, required using more than 
one initial guess in some instances, and did not improve 
the computation time observed with the pattern search 
me thod . 

Algorithm II does not add much complexity to Algorithm 
I, nor does it increase the computation time. This is be- 
cause most of the computation time is used for integration, 
and when the number of subintervals is increased, there is 
a corresponding decrease in the number of integration steps 
per subinterval and hence no significant change in computa- 
tion time occurs. 

B. PROBLEMS FOR FURTHER STUDY 

It would be worthwhile to investigate the application 
of the given algorithms to linear tracking problems because 
these problems have wide applications and methods for easi- 
ly realizable controllers for such systems are very desir- 
able. 

Another worthwhile extension is the application of the 
given methods to the design of simple compensators for 
automatic control systems. For example, for a third-order 
linear regulator with only one output, a second-order 
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compensator could be combined with the system and then in 
addition to the output feedback gain, the compensator para- 
meters could be optimized in a minimax sense for the over- 
all fifth-order system. 
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